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EXECUTIVE  SUMMARY 


The  purpose  of  this  study  is  to  examine  current  problems  of 
managing  competitive  prototype  programs.  An  illumination  of  these 
difficulties  may  be  helpful  to  defense  system  managers.  The 
examination  of  the  predicament  of  driving  both  competitors  to  a 
single  design  and  the  cross  flow  of  design  between  contractors  »vas 
used  as  a lead  question  for  the  examination  of  the  overall  area  of 
interest. 

Mr.  David  Packard,  former  Deputy  Secretary  of  Defense, 
formulated  his  policy  for  the  improvement  of  defense  systems 
acquisition  which  was  documented  in  DOD  Directive  5000. 1 published 
in  July  1971.  An  important  element  of  this  directive  was  the  develop 
ment  of  a strong  and  useable  technology  base.  This  base  was  to  be 
maintained  by  conducting  research  and  advanced  technology  efforts, 
including  prototyping. 

The  U.  S.  Air  Force  conducted  a study  in  the  spring  of  1971  to 
re-’iew  the  rationale  for  prototyping  and  to  structure  a sound  plan 
for  prototype  development.  It  identified  attractive  candidates  for 
prototyping  as  a demonstration  of  the  concept  of  obtaining  significant 
technological  advances  at  minimum  cost.  Two  of  these  candidates, 
the  Advanced  Medium  Short  Takeoff  and  Landing  Transport  and  the 
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Lightweight  Fighter  Program,  are  currently  in  competitive  prototype 
development.  In  that  these  two  programs  followed,  in  a classical 
sense,  the  guidance  for  prototyping,  they  were  used  as  the  primary 
source  of  information  concerning  current  problem  areas. 

Some  of  the  areas  investigated  were:  (1)  driving  both  competitors 
to  a single  design,  (2)  the  cross  flow  of  design  between  competitors, 
(3)  reduced  data,  (4)  excessive  visits  to  contractors'  facilities,  and 
(5)  limited  flight  or  system  test  hours. 

The  Source  Selection  Authority  has  a key  role  to  play  in  the  first 
two  problem  areas  mentioned  above.  During  the  evaluation  of  pro- 
posals, fundamentally  different  design  approaches  can  be  selected 
for  their  contribution  to  the  overall  objectives  of  the  program.  The 
program  director  is  able  to  initiate  the  program  with  more  than  one 
basic  design. 

The  examined  problem  areas,  while  of  concern  to  the  program 
directors,  did  not  reveal  any  unsurmountable  obstacles.  All  of  the 
areas  did  involve  some  lack  of  recognition  by  the  defense  management 
community  of  the  real  differences  in  the  approach  of  small  program 
offices,  reduced  funds  and  data,  and  the  lack  of  compliance  to  many 
production  oriented  directives. 

The  enthusiasm  of  the  participants  and  the  excellent  results  thus 
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PROBLEMS  OF  MANAGING  COMPETITIVE 
PROTOTYPE  PROGRAMS* 

Introduction 

The  problems  of  managing  competitive  prototype  programs 
are  not  well  documented.  This  is  due  in  a large  part  to  the  nature 
of  reduced  documentation  and  sniall  management  and  design  teams 
where  the  resolution  of  problems  is  accomplished  within  "closed" 
surroundings.  Prototyping  has  been  used  extensivoly  throughout 
DOD  in  the  past.  Recent  emphasis  by  Mr.  David  Packard,  former 
Deputy  Secretary  of  Defense,  stressed  the  need  for  improvement 
in  the  n-anagement  of  defense  system  acquisitions.  He  suggested 
prototyping  as  a means  for  reducing  uncertainty. 

Currently,  all  the  military  services  are  engaged  in  prototype 
programs.  Although  there  are  a variety  of  problems,  they  do  not 
appear  to  detract  from  the  goals  of  improved  acquisition. 
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CHAPTER  I 
Purpose  of  Study 

The  purpose  of  this  study  is  to  examine  some  of  the  management 
problems  associated  with  competitive  prototype  programs.  Although 
some  prototype  efforts  are  not  "competitive”  by  title,  the  strong 
possibility  of  an  eventual  production  contract  tends  to  make  all  proto- 
type developments,  involving  more  than  one  contractor,  competitive 
in  nature.  A complete  assessment  of  the  results  of  the  prototype 
approach  will  have  to  wait  for  the  completion  of  a number  of  the 
programs.  This  study  is  focused  on  the  problems  of  two  prototype 
developments  as  they  exist  during  the  fall  of  1973. 

Soon  after  taking  office,  Mr.  Packard  expressed  dissatisfaction 
with  the  total  package  procurement.  He  initiated  efforts  at  numerous 
levels  within  DOD  to  improve  the  overall  defense  systems  acquisition 
process.  He  contended  that  successful  system  development  required 
a strong  and  useable  technology  base,  developed  through  research 
and  advanced  technology  efforts  (1).  He  believed  the  DOD  should  put 
more  reliance  on  hardware  and  less  on  paper  studies  in  Advanced 
Development  (2).  Since  hardware  in  Advanced  Development  lends 
itself  to  prototyping,  there  has  been  a revival  of  strong  interest  in 
the  approach. 
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This  approach  is  not  an  altogether  new  concept  to  the  U.  S. 
military  departments.  One  of  the  earliest  examples  took  place 
shortly  after  the  War  of  1812  when  the  Army  became  interested  in 

a breech-loading  rifle.  As  a result  of  an  RFP or  maybe  an 

RFQ a number  of  contractors  submitted  actual  hardware 

examples  for  evaluation.  John  Hall's  design  was  accepted.  He  was 
given  an  order  for  a limited  production  and  these  rifles  were  issued 
to  the  only  rifle  regiment  in  the  Army  in  1820.  A test  and  evaluation 
was  conducted  in  Nebraska  and  the  rifles  were  found  to  be  unsatis- 
factory and  were  never  adopted  (3). 

During  the  first  half  of  this  century,  the  U.  S.  aircraft  industry 
used  prototypes  extensively.  The  military  services  have  descended 
from  some  great  prototyperes  --  Chanute,  the  Wrights,  Curtiss, 
Sikorsky,  Martin  and  more  recently  General  Dynamics,  Boeing, 
Douglas  and  McDonnell  (4). 

In  1962  the  Army  had  an  example  of  prototyping,  the  Light 
Observation  Helicopter  (LOH).  In  the  LOH  program,  the  Army  went 
out  with  a mission  type  description  only,  limited  the  contractors' 
responses  to  35  pages,  selected  two  winners,  and  instructed  them  to 
deliver  FAA  certified  flying  prototypes.  If  they  met  the  requirements, 
the  companies  were  to  be  paid.  A third  company  joined  in,  saying 
to  the  Army,  "Your  evaluation  team  said  it  was  impossible  to  do  what 
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we  proposed,  but  how  can  you  lose  if  we  don't  produce,  we  don't 
get  paid.  " The  results  of  this  effort  provided  three  good  sets  of 
prototypes  (3). 

In  the  spring  of  1971,  the  U.  S,  Air  Force  initiated  a study  to 
assess  the  prototype  concept.  It  was  directed  by  Brigadier  General 
Kenneth  R.  Chapman,  Deputy  Chief  of  Staff  Development  Plans, 
Headquarters  Air  Force  System  Command.  This  effort  reviewed 
the  rationale  for  prototyping  and  structured  a sound  plan  for  proto- 
type development.  Part  of  the  study  group  activity  was  devoted  to 
identifying  attractive  USAF  candidates  for  prototype  development 
as  a demonstration  of  the  concept  of  obtaining  significant  advances 
in  technology  at  minimum  cost.  The  resulting  plan  recommended 
streamlined  management  and  procurement  approaches  (5). 

During  the  same  general  time-frame,  the  U.  S.  Army,  recalling 
its  success  in  the  LOH,  initiated  an  experimental  prototype  program 
and  a development  prototype  program.  The  Heavy  Lift  Helicopter 
(HLH)  program  was  initiated  on  an  advanced  technology  component 
development  effort.  Later  it  was  converted  to  an  experimental  test 
bed  aircraft  to  demonstrate  that  the  technology  developed  in  the 
component  effort  was  in  hand  and  that  the  HLH  satisfied  the  mission 
requirements  prior  to  making  the  Engineering  Development  Decision. 
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The  Utility  Tactical  Transport  Aircraft  System  (UTTAS)  was 
established  as  a development  prototype  program  which  is  similar 
in  many  ways  to  the  earlier  LOH  program.  As  the  formulated 
replacement  for  the  UH-1  helicopter,  the  UTTAS  will  be  the  back- 
bone of  the  Army's  tactical  air  mobility. 

The  U.  S.  Air  Force  initiated  two  advanced  prototype  aircraft 
projects  in  Fiscal  Year  1972.  The  first  was  the  Lightweight 
Fighter  (LWF),  intended  to  demonstrate  advanced  fighter  technology 
specifically  designed  into  a small,  very  high  performance  aircraft, 
weighing  less  than  20,000  pounds.  The  second  was  the  Advanced 
Medium  Short  Takeoff  and  Landing  Transport  (AMST)  intended  to 
provide  data  on  the  cost  and  design  features  associated  with  short 
field  performance  in  an  aircraft  of  the  C-130  size  class.  A fully 
developed  AMST  could  be  a possible  replacement  for  the  aging  C-130 
airlift  force  (6). 

The  U.  S.  Navy  is  actively  engaged  in  a number  of  prototype 
programs  in  both  ships  and  aircraft.  Due  to  the  limited  scope  of 
this  study,  no  Navy  systems  are  incorporated  in  this  effort. 

Although  the  technological  aspects  of  these  programs  are  very 
important,  there  are  specific  goals  for  improved  program  manage- 
ment and  procurement.  Major  General  George  Sammet,  Jr.  , then 
Deputy  Chief  oi  Staff  for  Research  and  Development,  Department 
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of  the  Army,  addressed  two  of  these  goals  during  his  presentation 

to  the  Seminar  on  Prototyping  of  the  National  Security  Industrial 

Association,  23  - 24  February  1972; 

It  is  also  my  impression  that  Mr.  Packard's  real  goal  was  not 
prototyping  per  se.  He  was  as  much  interested  in  the  accom- 
panying management  techniques streamlined  as  they  were  - 

as  he  was  in  any  piece  of  hardware  which  may  resxilt  from 
prototyping.  Perhaps  even  more  so!  I think  he  was  using 
prototyping  as  a foil  to  cut  through  bureaucracy.  (3) 

The  goals  other  than  technological  will  be  addressed  more  fully 

due  to  their  pronounced  reoccurrence  during  interviews. 

Research  Questions 


In  order  to  organize  the  research  for  the  study  and  to  provide  a 
common  introduction  to  each  interview,  two  basic  research  questions 
were  selected  to  limit  my  research.  The  questions  are  related  to  the 
management  aspects  of  prototype  programs  where  more  than  one 
contractor  is  participating  iv  the  development.  The  two  research 
questions  are: 

(1)  How  do  you  prevent  driving  the  competitors  to  a single 
design  when  they  get  compared  in  detail  with  the  same  requirement? 

(2)  How  do  you  control  the  real  desire  of  a contractor  to 
quickly  incorporate  a change  that  will  match  a characteristic  in 
his  competitor's  design  that  he  feels  gives  his  competitor  an 
advantage? 


6 


Data  Collection  and  Analysis  Procedure 
As  mentioned  earlier,  there  are  very  few  recent  writings  on 
the  subject  of  prototype  management  problems,  a moderate  n\amber 
of  writings  on  the  initiation  of  the  prototype  concept  do  exist.  The 
personal  interview  approach  was  selected  to  obtain  current  informa- 
tion pertaining  to  currently  on-going  programs.  The  following 
personnel  were  interviewed:  Colonel  Robert  McDaniel,  for  an 
overall  view  of  the  subject  from  a DDR&E  viewpoint  on  6 August  1973; 
Brigadier  General  Leo  C.  Turner,  Project  Manager  UTTAS,  for  a 
program  with  a large  planned  production  run  on  9 August  1973; 

LTC  James  M.  Reed,  Jr. , Program  Director,  AMST,  and  Mr. 
Douglas  Ringwall,  Chief  Program  Control,  Light  Weight  Fighter, 
for  the  viewpoint  of  the  two  USAF  programs  that  resulted  from  the 
very  excellent  study  effort  conducted  by  the  USAF  on  19  October  1973. 

It  became  evident  from  the  first  interview  that  the  two  questions 
chosen  were  not  necessarily  the  areas  of  greatest  concern  to  the 
men  who  are  directly  involved  in  prototype  programs. 

While  personal  interviews  were  the  primary  mode  of  acquiring 
current  data,  a detailed  literature  search  was  conducted  to  include 
some  history  and  to  establish  the  overall  position  within  the  DOD 
concerning  prototyping. 
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Scope  and  Limitation 

The  scope  of  the  study  is  limited  to  detailed  consideration  of 
two  USAF  programs  for  the  interviews.  Numerous  attempts  were 
made  to  interview  the  Surface  Effects  Ship  project  office  personnel, 
but  the  nearness  of  Source  Selection  made  this  impossible.  The 
intent  was  to  cover  a program  soon  after  its  initiation,  and  this  was 
done  with  the  AMST.  The  other  program  was  to  be  one  well  along 
in  the  evaluation  of  its  prototypes.  The  Surface  Effects  Ship  program 
was  the  planned  program  for  this  area.  The  Light  Weight  Fighter 
with  first  flight  just  a few  months  away  made  a good  substitute  for 
this  area. 

The  study  looks  at  a number  of  problem  areas  other  than  the 
two  on  which  the  effort  was  centered.  These  problems  are: 

What  are  the  problems  in  selecting  people  for  the 
prototype  program  offices? 

What  are  the  problems  of  page  limited  RFPs  and 
responses? 

What  are  the  problems  of  reduced  reporting  and  the 
scarcity  of  data  in  the  program  office? 

What  are  the  problems  with  higher  headquarters  that 
still  expect  the  same  amount  of  briefings,  visits  to  the 
contractor's  facilities  and  responses  to  numerous  "what 
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ifs”  that  do  not  really  pertain  to  the  prototype 
system? 

What  are  the  problems  in  accommodating  all  the 
Developmental  tests  and  Operation  tests  that  higher 
headquarters  want  accomplished  even  though  a very 
few  systems  are  fabricated? 

These  areas  were  covered  in  varying  depths  depending  on  the 
amount  of  concern  voiced  by  the  persons  interviewed.  As  mentioned 
earlier,  the  first  two  problem  areas  were  methodically  addressed 
and  then  the  interviewees  responded  to  what  they  felt  were  problem 
areas  in  their  programs. 

Organization  of  the  Study 

The  study  is  presented  in  five  chapters  with  a suppor^g 
bibliography. 

Chapter  I explains  the  purpose  of  the  study  and  provides  the 
necessary  background  material  to  set  the  stage  for  the  particular 
areas  covered  by  the  study.  Unfamiliar  terms  are  defined.  The 
reader  is  assumed  to  have  a working  knowledge  of  the  terminology 
of  defense  system  acquisition. 

Chapter  II  reviews  research  which  is  related  to  the  study 
subject.  There  are  a number  of  public  statements  by  various  DOD 
officials  concerning  prototyping  and  the  rationale  supporting  this 
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method  of  development.  However,  there  is  very  little  published 
by  those  who  are  managing  their  programs. 

Chapter  III  addresses  the  first  interview  question.  There  are 
a number  of  techniques  available  to  prevent  driving  competitors  to 
a single  design. 

Chapter  IV  concerns  the  second  interview  question.  The  cross 
flow  of  design  is  not  as  detrimental  as  first  envisioned.  Proper 
external  controls  bound  the  problem  quite  well. 

Chapter  V contains  the  most  common  problem  areas  surfaced 
during  the  interviews.  The  responsible  manager  saw  them  as 
areas  of  concern  and  awareness  rather  thau  insurmountable  obstacles. 

Chapter  VI  is  a summary  of  the  study  and  the  author's  con- 
clusions. 
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CHAPTER  II 


Review  of  Related  Research 

The  numerous  speeches,  congressional  testimonies,  and  papers 
presented  to  professional  societies  center  on  two  practices  within 
the  defense  system  acquisition  process  that  have  led  to  excessive 
costs  and  unsatisfactory  results.  One  is  the  excessive  reliance 
on  paper  studies  and  paper  analysis.  The  problems  of  this  reliance 
have  been  evident  in  all  stages  of  past  programs,  conceptual,  valida- 
tion, scale  development  and  production.  The  other  problem  is  the 
concurrency  between  development  and  production  (6). 

The  prototype  approach  should  help  to  minimize  these  two 
problems.  The  underlying  objective  of  prototyping  is  to  place  more 
reliance  on  the  performance  of  hardware  and  less  reliance  on  paper 
analysis.  The  ability  to  evaluate  both  the  feasibility  and  utility  of  a 
new  system  before  production  and  in  some  cases  before  full-scale 
development  is  very  desirable  from  a defense  point  of  view  (6). 

Directives  and  paper  analyses  do  not  manage  programs;  people 
do.  In  that  there  is  very  little  written  on  the  "how"  of  managing 
prototype  efforts,,  it  is  critically  important  to  select  the  right  kind 
of  people  to  manage  these  developments,  both  within  DOD  and  within 
the  contractor  organization.  Ideally,  these  people  should  be 
experienced  in  the  thinking  involved  in  how  to  best  meet  the  objective 
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rather  than  dogged  compliance  to  directives.  In  that  decisions 
based  on  judgment  rather  than  detailed  analyses  will  be  needed 
many  times,  experience  on  which  to  base  intuitive  judgments  is 
most  important.  Effective  communications,  a winning  spirit  and 
trust  must  permeate  the  entire  management  complex,  government 
and  contractor  (4). 

In  that  there  were  a number  of  successful  prototype  programs 
prior  to  the  paper  study  and  analysis  era  of  the  1960's,  there  are 
good  reasons  to  expect  some  very  good  results  from  current  efforts 
in  this  field. 
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CHAPTER  III 

How  to  Prevent  Driving  Competitors 
to  a Single  Design 

The  Source  Selection  Authority  (SSA)  can  play  a significant  role 
in  this  area.  During  the  evaluation  of  the  contractor's  proposals, 
fundamentally  different  design  approaches  should  be  evaluated  on 
their  contribution  to  the  overall  objectives  of  the  program.  When 
sufficient  proposals  are  received,  the  SSA  can  avoid  duplicating 
similar  designs  by  picking  a winner  from  each  similar  design  group. 

An  example  would  be  picking  both  a single  engine  and  a twin  engine 
design  for  a fighter  aircraft.  When  criteria  such  as  this  is  used, 
the  contractors  must  be  told  so  that  the  fundamental  differences  do 
not  get  redesigned  out  of  the  program  (7:8). 

Since  prototype  programs  should  be  evaluated  against  criteria 
in  consonance  with  program  objectives,  performance  goals  rather 
than  detailed  specifications  should  be  the  key.  Envoking  specifications 
can  very  often  drive  the  design  closer.  A word  of  caution  at  this  point: 
the  specification  business  is  dramatically  different  in  an  experimental 
or  developmental  prototype  program  versus  a production  program. 

The  number  of  applicable  directives  can  be  reduced  by  an  order  of 
magnitude  when  there  is  no  requirement  to  be  prepared  for  production. 
This  aside,  the  program  manager  can  often  do  well  to  stay  out  of 


evaluating  changes  and  let  the  design  team  do  what  makes  sense  to 
them.  When  a design  change  can  be  made  within  fiscal  constraints, 
the  tradeoff  should  be  in  the  hands  of  the  design  team  (7:8). 

After  the  prototypes  are  fabricated,  the  defense  management 
team  must  guard  against  adding  features  to  one  competitor's  design 
that  they  have  found  to  their  liking  while  evaluating  the  other  design. 
The  tendency  for  the  cultist  to  pressure  this  change  is  very  real. 

An  example  of  this  is  finding  a rvinning  time  recorder  on  an 
auxiliary  power  unit  and  then  requiring  the  other  contractor  to  do 
the  same  by  government  direction  (5:9). 
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How  to  Dampen  the  Cross  Flow 
of  Design  Between  Competitors 

The  reduced  data  requirements  in  prototype  programs  provide 
a healthy  underlying  condition  to  stymie  the  cross  flow.  First,  by 
reducing  the  technical  data  that  is  reported,  knowledge  of  the  com- 
petitor's detailed  design  is  reduced  throughout  the  management 
community.  In  both  the  AMST  and  LWF  programs  the  periodic 
reporting  is  mostly  financial.  This  is  particularly  true  in  the  case 
of  month./  reports.  Secondly,  the  distribution  of  all  reports  to 
higher  headquarters  is  dramatically  reduced  in  these  programs. 
Without  knowledge  of  what  is  happening  <n  design,  the  functional 
areas  and  cultists  are  at  a disadvantage  in  being  able  to  discuss 
things  in  detail  and  thereby  provide  the  many  sources  where  con- 
tractor intelligence  can  find  out  the  details  of  the  competition's 
design.  The  LWF  program  was  on  contract  for  eight  months  before 
their  higher  headquarters  asked  for  any  significant  details  (8). 

Cross  flow  is  greatly  inhibited  if  the  program  manager  makes 
a concentrated  e'fort  to  suppress  his  own  people  from  picking  favorites 
among  the  competitors.  This  suggests  that  having  two  teams  in  the 
program  office,  one  to  monitor  each  contractor,  would  present  some 
potential  hazards.  As  in  the  case  of  the  first  question,  many  of  the 
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same  cautions  apply  here.  Government  personnel  who  talk  to  the 
contractor  must  be  exceedingly  careful  with  their  comments. 

Where  cross  flow  is  to  the  benefit  of  the  government,  it  should  not 
be  eliminated  but  it  should  not  originate  within  the  government 
team.  This  has  not  become  a problem  in  the  eyes  of  the  AMST  and 
LWF  program  managers  (7:8). 
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CHAPTER  V 


Other  Problem  Areas  as  Seen  by 
Prototype  Managers 

Indicative  of  the  difficulty  in  learning  about  the  probJems  of 
managing  competitive  prototypes  is  the  limited  response  to  the 
two  questions  generated  by  the  literature  research.  This  chapter 
addresses  the  problem  areas  as  seen  by  the  manager  currently 
running  prototype  programs. 

Explaining  What  You  Are  Doing 
Because  prototype  programs  with  drastically  reduced  data 
requirements  have  not  been  common  place  for  10-15  years, 
numerous  requests  are  put  on  the  prototype  officer  for  data  that 
just  doesn't  exist.  Also,  the  small  contractor  design  team  canuot 
afford  to  be  diverted  to  generating  the  data  without  having  serious 
impacts  on  the  program.  Whenever  a service  develops  a new  policy 
to  be  applied  to  all  systems,  the  prototype  program  manager  often 
must  try  to  explain  why  he  cannot  comply.  Again,  most  of  the 
directives,  the  majority  of  which  apply  to  production  or  eventual 
production,  which  emanate  from  the  functional  areas  are  not,  and 
appropriately  so,  applicable  to  prototype  programs.  Putting  this 
widget  and  that  widget  into  the  system  is  not  in  keeping  with  good 
prototype  contracts  (7:8). 
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Keeping  Visitors  Out  of  the  Contractor's  Facilities 
"I  am  going  to  be  in  the  area  on  this  other  program,  it  would 
be  a good  time  to  take  a look  at  that  prototype  program.  ” This 
accounts  for  most  of  the  communication  from  higher  headquarters. 
The  financial  arrangements,  small  design  team,  and  somewhat  of 
a hands-off  policy  do  not  support  the  normal  amount  of  government 
visitors.  This  is  a real  problem  to  both  the  AMST  and  LWF  pro- 
grams. Many  firm  and  polite  no's  seem  to  be  the  only  workable 
approach  (7:8). 

Problem  in  Transitioning  to  Production 
A combination  of  not  invoking  the  numerous  production  specifi- 
cations on  the  contractors  and  very  small  program  offices  would 
prevent  a rapid  transition  to  production.  Although  the  interviewees 
are  not  currently  faced  with  this  problem,  the  program  directors 
consider  it  one  of  great  potential  for  causing  all  kinds  of  problems 
for  a program.  Decision  makers  up  the  line  must  be  kept  aware  of 
this  potential  hazard  (7:8). 

Limited  Flight  Test  Hours 

The  contractor,  the  developer  and  the  user  have  many  objectives 
they  v/ish  to  satisfy  during  testing.  With  two  or  three  prototypes,  it 
is  clearly  impossible  to  accommodate  these  three  communities  in 
succession  or  with  partial  overlap.  The  AMST  program  is  planning 
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a joint  contractor,  flight  test  center,  user  flight  test  program. 

There  will  be  integrated  test  flight  crews  throughout  the  program. 
Still,  this  area  will  require  a great  deal  of  planning  and  some 
significant  concessions  on  the  part  of  each  of  these  three  agencies  (7). 

The  Schedule  is  Not  Sacred 

This  is  another  area  that  requires  education  of  the  higher 
headquarters.  The  costs  are  sacred,  the  schedule  is  not.  Both 
programs  recognize  that  explaining  why  two  contractors  are  at 
significantly  different  points  in  their  programs  is  very  difficult. 
Nonetheless,  they  also  feel  that  it  is  the  lesser  of  two  evils  and  that 
the  schedule  must  have  some  flexibility,  and  that  the  costs  should 
not  (7:8). 

Government  Furnished  Equipment 
Although  Government  Furnished  Equipment  (GFE)  is  not  a 
problem  at  all  peculiar  to  prototypes,  it  is  at  least  as  bad  here  as 
elsewhere.  The  AMST  program  manager  has  run  into  the  problem 
of  timely  GFE  delivery  in  the  first  year  of  his  program  (7). 

Quantitative  Versus  Subjective  Evaluations 
The  use  of  a quantitative  evaluation  where  the  proposals  are 
being  compared  to  performance  goals  rather  than  detailed  specifi- 
cations presents  a problem.  It  is  difficult  to  relate  highly  structured 
n\imerical  ratings  to  broad  statements  of  performance  goals.  The 
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LWF  program  manager  was  very  pleased  with  the  use  of  a sub- 
jective evaluation  in  line  with  the  philosophy  contained  in  the  USAF 
Prototype  Study  (7:8). 

A Good  Problem  Avoidance 

A major  problem  avoidance  experienced  by  both  the  AMST  and 
LWF  programs  is  included  here.  It  was  heartily  endorsed  by  each 
program  manager.  By  establishing  the  program  budgets  over  each 
year  of  the  program  and  telling  the  contractor  what  he  would  get 
each  year,  much  better  planning  was  accomplished.  Additionally, 
the  contractors  were  told  that  their  budget  was  set.  There  would 
be  no  attempts  to  negotiate  the  contractor  down  to  a lower  cost  than 
that  which  had  been  announced  (7:8), 

This  chapter  has  treated  these  problems  briefly.  There  is  not 
a completed  program  where  their  weight  could  be  evaluated  on  the 
overall  program.  It  appears  that  these  two  program  managers  are 
following  the  recommendations  of  the  Final  Report;  USAF  Prototype 
Study.  One  comment  from  the  study  comes  to  mind  time  and  again 
when  talking  to  these  people: 

"Personal  attention  as  to  what  is  going  on  must  be  the  accepted 
substitute  for  formal  data  and  reports.  " 

The  AMST  and  LWF  teams  are  characterized  by  the  above  quote. 
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CHAPTER  VI 


Summary  and  Conclusions 

The  purpose  of  this  study  was  to  examine  the  problems  of 
managing  competitive  prototype  programs.  The  illtimination  of 
some  of  these  problems  and  how  current  managers  see  them  in 
relation  to  their  programs  will  hopefully  be  helpful  to  defense 
system  managers  engaged  in  prototyping.  Chapters  III,  IV  and  V 
report  on  the  more  significant  problems  uncovered  by  this  effort. 
Although  these  problems  warrant  consideration  by  managers,  none 
of  the  personnel  interviewed,  nor  in  fact  the  literature  on  the 
subject,  revealed  any  insurmountable  obstacles. 

The  problems  of  driving  both  competitors  to  a single  design 
and  the  cross  flow  of  design  between  competitors  did  not  rank 
highly  in  difficulty  with  those  interviewed.  It  is  believed  this  was 
due  in  part  to  their  early  recognition  of  the  potential  for  problems 
in  these  areas  and  the  positive  steps  taken  to  avoid  them. 

The  problems  of  explaining  what  you  are  doing  and  keeping 
visitors  out  of  the  contractor's  facilities  should  diminish  as  DOD 
gains  more  experience  with  prototype  programs.  The  nvunber  of 
key  personnel  changes  at  the  top  levels  of  OSD  may  well  be  mitigating 
against  high  level  help  in  these  areas.  These  two  problem  areas  will 
persii?t  as  shortcomings  of  a large  bureaucratic  organization  that  is 
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unlikely  to  experience  any  real  big  change  in  the  near  future. 

Support  of  the  program  managers  in  their  decisions  in  these  areas 
would  provide  the  assistance  and  stability  required  to  keep  these 
problems  manageable. 

Limited  flight  or  system  test  hours  to  satisfy  a wide  variety  of 
reqmrements  will  continue  to  pose  major  problems  to  the  program 
manager.  There  are  too  many  agencies  that  can  present  require- 
ments in  this  area,  yet  none  of  them  can  ultimately  provide  the 
time  or  the  funds  that  would  be  required  to  provide  an  ideal  amount 
of  test  hours.  While  the  United  States  Congress  supports  many  of 
these  test  objectives,  they  also  have  the  final  authority  on  funds. 

The  past  performance  of  the  Senate  Armed  Services  Committee  has 
been  to  not  increase  either  the  time  allowed  or  the  funds  to  buy 
more  prototype  systems.  Joint  test  teams  working  on  integrated 
test  plans  will  provide  some  relief.  It  will  remain  the  program 
manager's  problem  to  establish  the  balance  between  desired  test 
requirements  and  resources  to  accomplish  these  test  requirements. 

The  remaining  problem  areas  do  not  present  the  magnitude 
of  difficulties  discussed  above.  They  should  be  overcome  by  early 
recognition  and  positive  steps  to  minimize  their  .npact. 

The  business  of  establishing  the  budgets  and  holding  them 
constant  is  particularly  important  in  that  it  supports  much  better 
planning.  With  the  better  planning,  the  design  teams  focus  on  the 
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basic  problems  internal  to  the  program  and  minimize  the 
external  "what  if"  drills  that  are  too  costly  in  time  and  money 
for  programs  on  modest  budgets  with  small  design  and  management 
teams. 

For  efficiency,  prototype  programs  must  be  managed  with  a 
minimum  of  constraints.  They  should  be  designed  to  meet  per- 
formance goals,  not  detailed  specifications. 

Competitive  prototyping  is  good.  Competition  is  the  real 
motivation  for  excellence;  the  contractor's  competition  is  a more 
efficient  "watch  dog"  than  any  government  program  office  and 
layers  of  high  headquarters.  The  DOD  should  realize  most  of 
the  goals  for  prototyping  held  by  the  OSD. 

Recommended  Areas  for  Future  Study 
Since  both  the  AMST  and  LWF  programs  are  still  in  the  design 
phase,  final  assessments  are  not  practical.  At  the  completion  of 
these  two  programs,  the  following  area  should  be  examined: 

What  were  the  major  management  problems  throughout  the 
life  of  the  program? 

Prior  to  the  completion  of  these  two  programs  the  following 
area  should  be  examined: 

What  should  be  the  major  considerations  in  transitioning  from 
an  advanced  technology  prototype  program  to  the  award  of  a 
production  contract? 
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APPENDIX 


Definition  of  Terms 


Advanced  Prototype  - A prototype  developed  in  an  Advanced 
Development  Program,  using  advanced  development  funds.  In  the 
following  descriptions  of  prototypes  by  Dr.  John  S.  Foster,  advanced 
prototypes  fall  into  the  first  two  categories,  but  not  the  third. 

(1)  Experimental  prototypes  can  be  used  to  explore 
promising  theories  or  laboratory  findings,  or  .to  bridge  the  gap 
between  theory  and  application  in  cases  where  the  technology 
is  "too  green"  for  direct  application  or  meaningful  cost  estimates. 
Examples  include  high  energy  laser  systems  and  the  so-called 
supercritical  wing  that  extends  the  limits  of  the  subsonic  flight 
regime. 


(2)  Developmental  prototypes  can  be  used  where  the 
government  lacks  either  the  confidence  or  the  urgency  to  enter  a 
system  into  full-scale  development.  The  purpose  may  be  to  gain 
information  concerning  cost  or  schedule  tradeoffs,  to  reduce 
development  lead  times  without  actually  developing,  or  to  reduce 
technological  or  manufacturing  uncertainties.  Examples  include 
the  B-1  bomber  program,  and  the  AWACS  radar,  and  F-15  avionics. 

(3)  Production  prototypes  can  be  used  when  high- rate, 
volume  production  is  planned  to  prove  the  system,  tools,  and 
production  methods.  This  has  been  the  most  common  form  of 
prototyping  and,  among  the  three  categories,  the  one  that  resembles 
most  the  eventual  production  system  in  terms  of  dimension, 
performance,  and  features  ( 10). 

AMST  - Acronym  for  Advanced  Medium  Short  Takeoff  and 
Landing  Transport 

HLH  - Acronym  for  Heavy  Lift  Helicopter 

LOH  - Acronym  for  Light  Observation  Helicopter 

LWF  - Acronym  for  Lightweight  Fighter 

SES  - Acronym  for  Surface  Effects  Ship 

UTTAS  - Acronym  for  Utility  Tactical  Transport  Aircraft  System 
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